On Identification of Sparse Multivariable ARX Model: A Sparse Bayesian Learning Approach

Although system identification of complex networks is widely studied, most of the work focuses on modelling the dynamics of the ground truth without exploring the topology because in many applications, the network topology is known as a priori. For instance, in industry since the system is artificial, its topology is fixed and the system dynamics is identified for the purpose of control. Nevertheless, in other cases, especially natural sciences, finding the correct network topology sometimes is even more important than modelling potential dynamics since it helps scientists understand the underlying mechanism of biological systems. More importantly, one of the most crucial features of such networks is sparsity which causes a huge difference if no effort is put on exploring the sparse structure. To fill the gap, this paper begins with considering the identification of sparse linear time-invariant networks described by multivariable ARX models. Such models possess relatively simple structure thus used as a benchmark to promote further research. With identifiability of the network guaranteed, this paper presents an identification method that infers both the Boolean structure of the network and the internal dynamics between nodes. Identification is performed directly from data without any prior knowledge of the system, including its order. The proposed method solves the identification problem using Maximum a posteriori estimation (MAP) but with inseparable penalties for complexity, both in terms of element (order of nonzero connections) and group sparsity (network topology). Such an approach is widely applied in Compressive Sensing (CS) and known as Sparse Bayesian Learning (SBL). We then propose a novel scheme that combines sparse Bayesian and group sparse Bayesian to efficiently solve the problem. The resulted algorithm has a similar form of the standard Sparse Group Lasso (SGL) while with known noise variance, it simplifies to exact re-weighted SGL. The method and the developed toolbox can be applied to infer networks from a wide range of fields, including systems biology applications such as signaling and genetic regulatory networks.